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ABSTRACT 

In  light  of  the  Office  of  the  Secretary  Defense’s  Roadmap  for  unmanned  aircraft  systems  (UASs),  there  is  a 
critical  need  for  research  examining  human  in  teraction  with  heterogeneous  unmanned  vehicles.  The  OSD 
Roadmap  clearly  delineates  the  need  to  investigate  the  “appropriate  conditions  and  requirements  under 
which  a  single  pilot  would  be  allowed  to  control  multiple  airborne  UA  [unmanned  aircraft] 
simultaneously  Towards  this  end,  in  this  paper,  we  provide  a  meta-analysis  of  research  studies  across 
unmanned  aerial  and  ground  vehicle  domains  that  investigated  single  operator  control  of  multiple 
vehicles.  As  a  result,  a  hierarchical  control  model  for  single  operator  control  of  multiple  unmanned 
vehicles  (UV)  is  proposed  that  demonstrates  those  requirements  that  will  need  to  be  met  for  operator 
cognitive  support  of  multiple  UV  control,  with  an  emphasis  on  the  introduction  of  higher  levels  of 
autonomy.  The  challenge  in  achieving  effective  management  of  multiple  UV  systems  in  the  future  is  not 
only  to  determine  if  automation  can  be  used  to  improve  human  and  system  performance,  but  how  and  to 
what  degree  across  hierarchical  control  loops,  as  well  as  determining  the  types  of  decision  support  that 
will  be  needed  by  operators  given  the  high  workload  environment.  We  address  when  and  how  increasing 
levels  of  automation  should  be  incorporated  in  multiple  UV  systems  and  discuss  the  impact  on  not  only 
human  performance,  but  more  importantly,  system  performance. 

Index  Terms — multiple  unmanned  aerial  vehicles,  supervisory  control,  operator  capacity 


1.0  INTRODUCTION 

Unmanned  aerial  vehicles  (UAVs)  are  quickly  becoming  ubiquitous  in  military  command  and  control 
operations.  With  reduced  radar  signatures,  increased  endurance  and  the  removal  of  humans  from 
immediate  threat,  unmanned  (also  known  as  uninhabited)  aerial  vehicles  have  become  indispensable  assets 
to  militarized  forces  around  the  world,  as  proven  by  the  extensive  use  of  the  Shadow  and  Predator  in 
recent  conflicts.  Despite  the  absence  of  a  crew  onboard  any  of  these  UAVs,  human  operators  are  still 
needed  for  supervisory  control. 

UAVs  require  human  guidance  to  varying  degrees  and  often  through  several  operators,  which  is  what 
essentially  defines  a  UAS  (Unmanned  Aerial  System).  For  example,  the  Predator  and  Shadow  each  require 
a  crew  of  two  to  be  fully  operational.  However,  with  current  military  focus  on  streamlining  operations  and 
reducing  manning,  there  has  been  an  increasing  effort  to  design  systems  such  that  the  current  many-to-one 
ratio  of  operators  to  vehicles  can  be  inverted.  In  light  of  the  Office  of  the  Secretary  of  Defense  Roadmap 
for  UASs  [1],  there  is  a  critical  need  for  research  examining  human  interaction  with  multiple  UASs,  which 
has  also  recently  been  identified  as  an  essential  need  by  the  Committee  on  Autonomous  Vehicles  in 
Support  of  Naval  Operations  [2]. 

In  response  to  this  need,  this  paper  first  defines  human  supervisory  control  of  one  and  multiple  UAVs.  It 
then  analyzes  past  literature  to  examine  potential  trends  in  supervisory  control  research  of  multiple  UAVs. 
Specific  attention  is  paid  to  automation  strategies  for  decision-making  and  action  as  levels  of  embedded 
automation  significantly  differ  between  applications.  We  will  demonstrate  that  as  autonomy  increases 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1 .  REPORT  DATE  2.  REPORT  TYPE 

SEP  2009  N/A 

4.  TITLE  AND  SUBTITLE 

Automation  Architecture  for  Single  Operator-Multiple  UAV 
and  Control 

6.  AUTHOR(S) 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Humans  and  Automation  Laboratory  Massachusetts  Institute  of 
Technology  Cambridge,  MA 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

See  also  ADB381581.  RTO-EN-SCI-208  Advanced  Automation  Issues  for  Supervisory  Control  in 
Manned-Unmanned  Teaming  Missions  (Problemes  de  l’automatisation  evoluee  pour  la  supervision  de 
missions  associant  des  systemes  habites  et  non  habites),  The  original  document  contains  color  images. 

14.  ABSTRACT 

In  light  of  the  Office  of  the  Secretary  Defenses  Roadmap  for  unmanned  aircraft  systems  (UASs),  there  is  a 
critical  need  for  research  examining  human  interaction  with  heterogeneous  unmanned  vehicles.  The  OSD 
Roadmap  clearly  delineates  the  need  to  investigate  the  appropriate  conditions  and  requirements  under 
which  a  single  pilot  would  be  allowed  to  control  multiple  airborne  UA  [unmanned  aircraft]  simultaneously. 
Towards  this  end,  in  this  paper,  we  provide  a  meta-analysis  of  research  studies  across  unmanned  aerial 
and  ground  vehicle  domains  that  investigated  single  operator  control  of  multiple  vehicles.  As  a  result,  a 
hierarchical  control  model  for  single  operator  control  of  multiple  unmanned  vehicles  (UV)  is  proposed  that 
demonstrates  those  requirements  that  will  need  to  be  met  for  operator  cognitive  support  of  multiple  UV 
control,  with  an  emphasis  on  the  introduction  of  higher  levels  of  autonomy.  The  challenge  in  achieving 
effective  management  of  multiple  UV  systems  in  the  future  is  not  only  to  determine  if  automation  can  be 
used  to  improve  human  and  system  performance,  but  how  and  to  what  degree  across  hierarchical  control 
loops,  as  well  as  determining  the  types  of  decision  support  that  will  be  needed  by  operators  given  the  high 
workload  environment.  We  address  when  and  how  increasing  levels  of  automation  should  be  incorporated 
in  multiple  UV  systems  and  discuss  the  impact  on  not  only  human  performance,  but  more  importantly, 
system  performance. 
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across  hierarchical  control  loops,  the  number  of  vehicles  a  single  operator  can  control  also  increases. 
However,  as  we  will  discuss,  increasing  system  autonomy  can  introduce  negative  consequences  in  term  of 
operator  situation  awareness  and  complacency.  The  focus  of  this  paper  is  primarily  on  human  supervisory 
control  of  multiple  UASs,  with  a  discussion  of  broader  command  and  control  implications.  However,  the 
reader  is  referred  to  [3-6]  for  more  detailed  discussions  about  other  socio-technical  implications  of 
automation  and  network-centric  operations  in  command  and  control  domains. 

2.0  SUPERVISORY  CONTROL  OF  MULTIPLE  UAVS 

The  move  from  platform-centric  warfare  to  Network  Centric  Warfare  (NCW)  represents  a  shift  in  the  role 
of  humans  both  in  mission  planning  and  actual  operation.  As  has  already  been  evidenced  in  the 
development  of  fly-by-wire,  highly  automated  aircraft  and  missile  systems  (such  as  Tomahawk  and 
Patriot),  military  operators  are  less  in  direct  manual  control  of  systems,  but  more  involved  in  the  higher 
levels  of  planning  and  decision-making  and  remote  operations.  This  shift  in  control  from  lower  level  skill- 
based  behaviors  to  higher  level  knowledge -based  behaviors  is  known  as  human  supervisory  control 
(HSC).  HSC  is  the  process  by  which  a  human  operator  intermittently  interacts  with  a  computer,  receiving 
feedback  from  and  providing  commands  to  a  controlled  process  or  task  environment,  which  is  connected 
to  that  computer  (Figure  1)  [7].  All  UAVs  in  the  DoD  inventory  operate  at  some  level  of  supervisory 
control  as  depicted  in  Figure  1 . 


Figure  1 :  Human  Supervisory  Control  [3] 


Human  supervisory  control  in  UAV  operation  is  hierarchical,  as  represented  in  Figure  2.  The  innermost 
loop  of  Figure  2  represents  the  basic  guidance  and  motion  control  loop  which  is  the  most  critical  loop  that 
must  obey  physical  laws  of  nature  such  as  aerodynamic  constraints  for  UAVs.  In  this  loop,  operator 
actions  are  focused  only  on  short  term  and  local  control  (keeping  the  aircraft  in  stable  flight),  and 
generally  human  control  in  this  loop  requires  skill-based  behaviors  that  rely  on  automaticity  [8]. 
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Figure  2:  Hierarchical  Control  Loops  for  a  Single  UAV 

The  second  loop,  the  navigation  loop,  represents  the  actions  that  some  agent,  whether  human  or 
computer-driven,  must  execute  to  meet  mission  constraints  such  as  routes  to  waypoints,  time  on  targets, 
and  avoidance  of  threat  areas  and  no-fly  zones.  The  outermost  loop  represents  the  highest  levels  of 
control,  that  of  mission  and  payload  management.  In  this  loop,  sensors  must  be  monitored  and  decisions 
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made  based  on  the  incoming  information  to  meet  overall  mission  requirements.  In  this  loop,  decisions 
require  knowledge-based  reasoning  that  includes  judgment,  experience,  and  abstract  reasoning  that  in 
general  cannot  be  performed  by  automation. 

Finally,  the  system  health  and  status  monitoring  loop  in  Figure  2  represents  the  continual  supervision 
that  must  occur,  either  by  a  human  or  automation  or  both,  to  ensure  all  systems  are  operating  within 
normal  limits.  The  control  loop  line  is  dashed  as  is  represents  a  highly  intermittent  loop  in  terms  of  the 
human,  i.e.,  if  the  human  is  engaged  in  another  task,  with  the  highest  priority  given  to  the  innermost  loop, 
health  and  status  monitoring  becomes  a  distant,  secondary  task. 

From  the  human-in-the-loop  perspective,  if  the  inner  loops  fail,  then  the  higher  (outer)  loops  will  also 
fail.  The  dependency  of  higher  loop  control  on  the  successful  control  of  the  lower  loops  drives  human 
limitations  in  control  of  a  single,  and  especially  so,  for  multiple  UAVs.  If  humans  must  interact  in  the 
guidance  and  motion  control  loop  (hand  fly  a  UAV),  the  cost  is  high  because  this  effort  requires 
significant  cognitive  resources.  What  little  spare  mental  capacity  is  available  must  be  divided  between  the 
navigation  and  mission  management  control  loops.  Violations  of  the  priority  scheme  represented  in  Figure 
2  have  led  to  serious  problems  exemplified  by  numerous  Predators  crashes.  When  operators  become 
cognitively  saturated  or  do  not  correctly  allocate  their  cognitive  resources  to  the  appropriate  control  loops 
in  the  correct  priorities,  they  violate  the  control  loops  constraints,  potentially  causing  catastrophic  failure. 
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While  Figure  2  demonstrates  supervisory  control  at  the  single  vehicle  level,  Figure  3  represents  a 
notional  system  architecture  that  will  be  required  for  single  operator  control  of  multiple  UASs.  In  order  to 
achieve  this  futuristic  system,  operators  will  need  to  interact  with  an  overall  mission  and  payload  manager, 
while  relegating  routine  navigation  and  motion  control  tasks  to  automation.  The  challenge  in  achieving 
effective  management  of  multiple  UAVs  in  the  future  is  not  only  to  determine  if  automation  can  be  used  to 
reduce  workload,  but  how  and  to  what  degree  in  each  of  the  control  loops  in  Figures  2  &  3,  as  well  as 
what  kinds  of  decision  support  will  be  needed  by  operators  given  the  high  workload  environment. 
Moreover,  the  depiction  of  a  single  mission  and  payload  management  system  for  multiple  vehicles  as  seen 
in  Figure  3  is  not  just  specific  to  UASs,  as  any  vehicle,  manned  or  unmanned  or  above  or  on  the  ground 
and  under  water,  can  be  a  node  under  a  common  mission  management  system.  Thus  research  that 
advances  single  operator-multiple  UAV  command  and  control  capabilities  will  actually  set  the  stage  for 
the  implementation  of  NCW  concepts  in  general. 
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3.0  LEVELS  OF  AUTOMATION 

Increasing  the  autonomy  across  the  three  control  loops  discussed  previously  (Figures  2  &  3)  is  the  critical 
architecture  component  for  allowing  one  or  a  small  team  of  operators  to  effectively  control  multiple 
UAVs.  By  increasing  UAS  autonomy,  operator  workload  will  theoretically  be  reduced  as  it  could  reduce 
the  number  of  tasks  for  the  operator,  and  it  should  reduce  the  level  of  interaction  even  at  the  highest  levels 
of  control  in  Figures  2  and  3.  For  example,  those  UAVs  that  are  flown  in  an  autopilot  mode  relieve  the 
operator  from  the  manual  flying  tasks  that  require  significant  cognitive  resources.  This  frees  the  operator 
to  perform  other  critical  tasks  like  mission  planning  and  imagery  analysis. 

Higher  levels  of  automation  across  the  control  loops  depicted  in  Figures  2  and  3  will  be  critical  in 
achieving  the  single  operator-multiple  UAV  control  vision,  but  how,  when,  where,  and  what  level  of 
automation  should  be  introduced  is  still  a  difficult  problem.  While  workload  mitigation  can  occur  through 
increasing  automation,  it  can  inadvertently  cause  higher  workload  as  well  as  loss  of  situational  awareness, 
complacency,  and  skill  degradation  [9].  For  example,  some  UAV  researchers  have  found  that  intermediate 
levels  of  management-by-consent  (automation  as  an  assistant  to  the  operator)  is  preferable  to  manual  or 
more  fully  automated  control  [10].  However,  management-by-consent  means  that  the  number  of  tasks 
could  be  high  since  operators  must  always  be  in  the  loop,  potentially  saturating  operators  especially  in  the 
multiple  UAV  domain.  Moreover,  as  has  been  shown  in  multiple  UAV  control  research,  operator 
performance  can  dramatically  decrease  under  management-by-consent  given  increasing  workload  and 
various  decision  aids  [11,  12]. 

Given  that  an  increasing  number  of  tasks  will  have  to  be  automated  to  achieve  single  operator  control  of 
multiple  UAVs,  the  question  then  becomes  what  to  allocate  to  automation.  Previous  research  has 
demonstrated  that  in  the  scheduling  and  execution  of  high-level  tasks,  of  multiple  UAVs,  management-by- 
exception  can  improve  operator  performance  [12].  Management-by-exception  occurs  when  automation 
decides  to  take  an  action  based  on  some  set  of  pre-determined  criteria,  and  only  gives  operators  a  chance 
to  veto  the  automation’s  decision.  While  this  control  scheme  can  be  effective  in  time -critical,  high  risk 
domains  like  shutting  down  a  near-critical  reactor,  in  intentional,  highly  uncertain  domains  like  command 
and  control,  it  can  be  dangerous.  Under  this  control  scheme,  operators  are  more  likely  to  exhibit 
automation  bias,  a  decision  bias  that  occurs  when  operators  become  over -reliant  on  the  automation  and  do 
not  check  to  ensure  automated  recommendations  are  correct  [13]. 

Automation  bias  was  operationally  seen  recently  in  the  2004  war  in  Iraq  when  the  U.S.  Army’s  Patriot 
missile  system,  operating  in  a  management-by-exception  mode,  engaged  in  fratricide,  shooting  down  a 
British  Tornado  and  an  American  F/A-18,  killing  three  aircrew.  The  system  was  designed  to  operate  under 
management-by-exception,  and  operators  were  given  approximately  15  seconds  to  veto  a  computer 
solution.  Unfortunately  the  displays  were  confusing  and  often  incorrect,  and  operators  were  admittedly 
lacking  training  in  a  highly  complex  system  [14].  Given  the  laboratory  evidence  that  given  an  unreliable 
system,  humans  are  still  likely  to  approve  computer-generated  recommendations  [15],  it  is  not  surprising 
that  under  the  added  stress  of  combat,  Patriot  operators  did  not  veto  the  computer’s  solution.  Automation 
bias  is  a  significant  concern  for  command  and  control  systems  so  it  will  be  critical  to  ensure  that  when 
higher  levels  of  automation  are  used,  especially  at  the  management -by-exception  level,  that  this  effect  is 
minimized. 

4.0  A  META-ANALYSIS  OF  PREVIOUS  MULTIPLE  UAV  STUDIES 

There  have  been  numerous  research  studies  published  that  have  examined  various  aspects  of  multiple 
UAV  control.  We  performed  a  meta-analysis  across  those  studies  that  focused  either  explicitly  on  operator 
capacity  or  human  supervisory  control  aspects  of  multiple  vehicle  control  in  order  to  determine  any 
significant  trends  or  lessons  learned,  particularly  in  regards  to  levels  of  automation  and  the  control  loops 
discussed  above. 
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4.1  Case  Study  Summaries 

One  solution  investigated  by  Dixon  et  al.  to  reduce  UAV  operator  workload  in  the  control  of  one  or  more 
(small)  UAVs  such  as  the  Shadow  consisted  of  adding  auditory  and  automation  aids  to  support  the 
potential  single  operator  [16].  They  showed  that  a  single  operator  could  theoretically  fully  control  a  single 
UAV  (both  navigation  and  payload)  if  appropriate  offloading  strategies  were  provided.  For  example,  aural 
alerts  improved  performance  in  the  tasks  related  to  the  alerts,  but  not  others.  Conversely,  it  was  also 
shown  that  adding  automation  benefited  both  tasks  related  to  automation  (e.g.  navigation,  path  planning, 
or  target  recognition)  as  well  as  non-related  tasks.  However,  their  results  demonstrate  that  human 
operators  may  be  limited  in  their  ability  to  control  multiple  vehicles  which  need  navigation  and  payload 
assistance,  especially  with  unreliable  automation.  These  results  are  concordant  with  the  single -channel 
theory,  stating  that  humans  alone  cannot  perform  high  speed  tasks  concurrently  [17,  18].  However,  Dixon 
et  al.  propose  that  reliable  automation  could  allow  a  single  operator  to  fully  control  two  UAVs. 

Reliability  and  the  related  component  of  trust  is  a  significant  issue  in  the  control  of  multiple  uninhabited 
vehicles.  Ruff  et  al.  [10]  found  that  if  system  reliability  decreased  in  the  control  of  multiple  UAVs,  trust 
declined  with  increasing  numbers  of  vehicles  but  improved  when  the  human  was  actively  involved  in 
planning  and  executing  decisions.  These  results  are  similar  to  those  found  by  Dixon  et  al.  in  that  systems 
that  cause  distrust  reduce  operator  capacity  [19]. 

In  addition,  Ruff  et  al.  [10,  20]  determined  that  higher  levels  of  automation  actually  degraded 
performance  when  operators  attempted  to  control  up  to  four  UAVs.  Results  showed  that  management-by- 
consent  (in  which  a  human  must  approve  an  automated  solution  before  execution)  was  superior  to 
management-by-exception  (where  the  automation  gives  the  operator  a  period  of  time  to  reject  the 
solution).  Management -by  consent  appeared  to  provide  the  best  situation  awareness  ratings,  the  best 
performance  scores,  and  the  most  trust  for  controlling  up  to  four  UAVs. 

Dunlap  et  al.  also  subscribe  to  management -by-consent  in  their  development  of  a  distributed  architecture 
to  control  multiple  unmanned  combat  aerial  vehicles  (UCAVs)  [21].  In  this  system,  a  UCAV  plan  is 
proposed  by  the  automation,  and  the  operator  can  either  accept  or  reject  the  plan  or  submit  an  alternative. 
This  recommendation  can  include  both  target  assignments  and  routing.  While  they  tested  4,  6,  and  8 
UCAVs  with  increasing  levels  of  environmental  complexity,  their  final  design  limited  the  UCAV  load  out 
at  four.  In  one  experiment,  they  noted  that  automation  bias  was  a  prevalent  problem,  stating  the  operators 
“had  become  attenuated  to  automatically  accepting  the  usually  correct  proposals  from  the  UCAVs,”  which 
resulted  in  an  increased  kill  rate  for  no-targets  under  the  higher  levels  of  automation. 

In  terms  of  actually  predicting  how  many  UAVs  a  single  operator  can  control,  there  are  only  a  few 
studies  that  examine  this  question.  Cummings  and  Guerlain  [22]  showed  that  operators  could 
experimentally  control  up  to  twelve  Tactical  Tomahawk  missiles  given  significant  missile  autonomy. 
Operators  only  had  to  interact  in  the  mission  management  loop  and  all  other  loops  were  highly  automated. 
In  a  UCAV  setting,  Cummings,  et  al.  [23,  24]  demonstrated  that  the  number  of  UCAVs  that  a  single 
operator  can  control  is  not  just  a  function  of  the  level  of  decision  support  automation,  but  also  the 
operational  tempo  and  demands.  Operators  under  low  workload  performed  well  regardless  of  the  level  of 
decision  support  but  under  high  workload,  performance  degraded.  When  considering  operational  and 
workload  demands,  for  a  suppression  of  enemy  air  defense  mission,  operator  capacity  was  estimated  at  5 
UCAVs. 

In  a  demonstration  of  the  capabilities  of  a  single  operator  attempting  to  control  multiple  Wide  Area 
Search  Munitions  (WASMs),  given  high  levels  of  autonomy  across  all  control  loops  in  Figure  3  with  only 
higher-order  goal  tasking  for  mission  management,  Lewis  et  al.  posit  that  an  operator  can  effectively 
control  up  to  eight  WASMs  [25].  The  assumption  is  that  the  automation  embedded  in  the  vehicles 
coordinates,  without  human  intervention,  specific  tasks  such  as  target  detection,  choice  of  the  most 
appropriate  member  to  execute  the  mission,  etc,  which  are  capabilities  that  are  not  yet  operational.  The 
WASM  study  is  similar  to  the  Tactical  Tomahawk  study  in  that  all  flight  control  and  navigation  functions 
are  allocated  to  the  automation  alone,  and  the  human  intervenes  for  very  high  level  goal  management. 

Thus  there  have  been  a  cross-section  of  studies  that  have  examined  operator  performance  and  capacity  in 
the  control  of  multiple  UAVs,  it  is  not  clear  how  any  meaningful  comparisons  can  be  made  across  the 
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different  domains  primarily  because  of  two  parameters,  1)  what  constitutes  control  and  2)  what  level  of 
automation  was  used  to  aid  the  operators?  In  order  to  more  directly  compare  these  studies,  the  following 
section  will  discuss  the  scale  on  which  comparisons  can  be  made. 

4.2  Level  of  Automation  Trends 

In  this  meta-analysis,  we  extracted  the  maximum  number  of  UAVs  that  an  operator  effectively  controlled 
in  each  study.  It  should  be  noted  that  in  all  of  these  reported  studies,  the  control  occurred  in  simulated  test 
beds  of  medium  to  high  fidelity.  We  identified  what  we  interpreted  to  be  the  approximate  levels  of 
automation  (LOAs)  across  the  control  loop(s)  from  Figure  3.  While  numerous  levels  and  scales  of 
automation  and  autonomy  have  been  proposed  [9,  26-28],  we  chose  the  ten  level  scale  originally  proposed 
by  Sheridan  and  Verplank  [29]  (SV  -  LOA),  as  this  is  a  commonly  referenced  taxonomy.  We  combined 
some  categories  in  Table  1  to  reflect  functional  similarities.  For  example,  levels  7-10  were  combined  since 
the  human  can  take  no  action.  Recognizing  that  different  stages  of  information  processing  can  be 
supported  by  automation  [9],  the  decision  and  action  selection  stage  is  represented  in  our  assessment. 

In  Table  2,  which  presents  a  summary  of 

these  findings,  the  numbers  of  UAVs  Table  1.  Levels  of  Automation 

potentially  controllable  by  a  single  human 
operator  are  referenced  along  with 
estimated  levels  of  automation  for  each  of 
the  three  control  loops  (MC  -  Motion 
Control  inner  loop,  N  -  Navigation,  MM  - 
Mission  Management  outer  loop.).  It 
should  be  emphasized  that  the  LOAs 
selected  were  approximate  since  they  were 
both  subject  to  interpretation  and  assigned 
post  hoc  from  studies  not  originally 
intended  to  answer  our  research  question. 

In  addition,  in  many  simulations,  the  LOA 
was  not  fixed  and  we  have  indicated  the 
range  of  LOAs  in  these  cases.  In  this 
comparison,  we  also  included  an  air  traffic 
control  (ATC)  study  since  it  embodies 
many  of  the  same  principles  of  human 
supervisory  control  that  are  relevant  to  the 
control  of  multiple  UAVs  [30].  Since  air 
traffic  controllers’  primary  focus  is  safe 
navigation  of  aircraft,  there  is  no  associated 
mission  management  control  loop. 

Table  2  reveals  interesting  trends. 

Without  explicitly  discussing  it  in  their 
respective  studies,  all  researchers  automated  the  inner  motion  control  loop  as  depicted  in  Figures  2  and  3. 
Thus  some  form  of  autopilot  was  needed  to  relieve  operator  workload  and  free  cognitive  resources  for 
higher  loop  control.  To  achieve  the  goal  of  one  person  controlling  many  UAVs,  operators  should  only 
monitor  the  piloting/maneuvering  of  the  vehicle,  not  do  it  themselves.  However  this  is  a  cultural  problem 
more  than  it  is  a  technological  problem,  as  this  technology  is  available  today  in  all  UAVs,  but  resisted  in 
some  communities,  i.e.,  some  organizations  still  insist  that  a  human  “fly”  the  vehicle  instead  of 
commanding  it  to  various  flight  profiles. 

In  addition,  Figure  4a  demonstrates  a  general  increasing  trend  in  the  number  of  vehicles  an  operator  can 
control  as  a  function  of  increasing  automation  in  the  navigation  control  loop  of  Figure  2.  Thus  given 
increasing  navigation  support  and  a  fully  autonomous  flight  control  system,  operators  can  handle  more 
UAVs  when  they  do  not  have  to  attend  to  local  and  even  global  navigation  concerns.  The  highest  operator 
capacity  was  seen  in  the  Tomahawk  missile  and  WASM  domains  since  because  as  one-way  UAVs 


SV- 

LOA 

Our 

LOA 

Automation  description 

1 

I 

The  computer  offers  no  assistance: 
human  must  take  all  decision  and 
actions. 

2 

II 

The  computer  offers  a  complete  set  of 
decision/action  alternatives. 

3 

III 

The  computer  offers  a  selection  of 
decisions/actions. 

4/5 

IV 

The  computer  suggests  one 
alternative,  and  executes  that 
suggestion  if  the  human  approves 
(management  by  consent) 

6 

V 

The  computer  suggests  one 
alternative  and  allows  the  human  a 
restricted  time  to  veto  before 
automatic  execution  (management  by 
exception). 

7/8/9/10 

VI 

The  human  is  not  involved  in  the 
decision-making  process,  the 
computer  decides  and  executes 
autonomously. 

1  -6 


RTO-EN-SCI-208 


UNCLASSIFIED/UNLIMITED 


UNCLASSIFIED/UNLIMITED 

Automation  Architecture  for  Single  Operator-Multiple  UAV 

Command  and  Control 


travelling  at  high  speeds,  there  is  Table  2.  Multiple  UAV  Study  Comparison 

little  time  and  fuel  for  human 
interaction  beyond  high-level 
goal  direction. 

When  examining  the  mission 
and  payload  management  control 
loop,  Figure  4b  demonstrates  the 
ability  for  operators  to  control 
more  vehicles  as  they  are 
provided  with  increasing 
automated  decision  support.  It  is 
interesting  to  note  that  given 
some  automated  navigation 
assistance  and  management-by- 
consent  automation  in  the 
mission  management  loop,  there 
is  a  convergence  of  operator 
capacity  at  4-5  vehicles  per  operator.  The  next  remarkable  increase  in  operator  capacity  (8-12  vehicles)  is 
not  seen  until  management-by-exception  is  introduced  in  the  mission  management  or  navigation  loops. 
These  increased  levels  of  automation  will  be  critical  for  increased  operator  capacity,  since  as  previously 
discussed,  if  operators  are  required  to  attend  to  local  navigation  functions,  they  simply  do  not  have  the 
cognitive  resources  to  successfully  attend  to  all  of  the  tasks  in  the  mission  and  payload  management  loop. 

One  important  consideration  not  evaluated  here  (primarily  due  to  lack  of  any  experimental  evidence)  is 
the  impact  of  heterogeneous  UAV  control.  While  all  the  studies  included  in  this  review  were 
homogeneous  (except  for  the  ATC  study),  the  future  of  UAV  operations  will  include  mixed  operations 
such  that  potentially  different  UAV  platforms  will  interact  in  the  same  airspace.  In  addition,  another 
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uv# 

MC 
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MM 
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Dixon  et  al.  [16]  (baseline) 

VI 

I 

I 

1 
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Dixon  et  al.  [16]  (auto-pilot) 

VI 

IV 
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2 

3 

Dixon  et  al.  [16]  (auto-alert) 

VI 

I 

IV 

2 

4 

Ruff  et  al.  [10,  20] 

VI 

IV-V 

IV 

4 

5 

Dunlap  [21] 

VI 

IV 

IV 

4 

6 

Cummings,  et  al.  [23,  24] 

VI 

IV 

III-IV 

5 

7 

Lewis  et  al.  [25] 

VI 

VI 

IV-V 

8 

8 

Cummings  &  Guerlain  [22] 

VI 

VI 

IV 

12 

9 

Hilbum  et  al.  [30]  (ATC  Study) 

VI 

V 

N/A 
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(a)  Navigation  Loop 


(b)  Mission  Management  Loop 


important  variable  is  that  operators  never  manage  multiple  vehicles  in  isolation  and  are  part  of  a  larger 
team  so  this  effort  needs  to  be  extended  to  collaborative  teaming  environments. 

Figure  4:  Max  Number  of  UAVs  vs.  LOAs  for  the  (a)  Navigation  and  (b)  Mission  Management 

Loops 
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Our  purpose  in  relating  prior  multiple  UAV  human-in-the-loop  studies  is  not  to  make  claims  about  a 
specific  theoretical  maximum  for  a  given  LOA,  but  rather  to  illustrate  the  possibility  that  with  increasing 
automation  in  the  motion  control  and  navigation  loops  and  collaborative  decision  support  for  the  mission 
management  loop,  it  is  likely  that  operators  can  control  an  increasing  number  of  UAVs.  Table  2  and 
Figure  4  represent  approximate  generalizations  and  both  automation  decision  support  and  subsequent 
display  design  can  significantly  alter  human  performance  (e.g.,  [11,  31]). 

4.3  Linking  Operator  Capacity  Estimates  with  Performance  Parameters 

One  of  the  limitations  common  across  the  previously  reviewed  studies  reviewed  is  the  lack  of  measurable 
performance  metrics.  In  general,  the  performance  of  the  operators  was  deemed  acceptable  as  a  function  of 
expert  observation,  which  is  a  valid  method  for  performance  assessment  [32]  but  is  not  generalizable 
across  domains  and  not  useful  for  a  priori  predictions  as  it  is  essentially  a  descriptive  operator  capacity 
prediction.  Thus  what  is  needed  is  some  kind  of  performance  metric  that  captures  both  aspects  of  human 
and  system  performance,  which  indicates  an  objective  level  of  goodness  and/or  satisficing  [33]  (i.e.,  a 
“good  enough”  solution  as  opposed  to  optimal.)  Indeed,  the  focus  on  key  performance  parameters  (KPPs) 
is  a  major  focus  for  the  Department  of  Defense,  particularly  in  terms  of  network  centric  command  and 
control  [34], 

Research  is  currently  underway  to  address  this  disconnect  between  operator  capacity  and  performance, 
which  has  resulted  in  the  development  of  many  possible  KPPs.  A  recent  study  demonstrated  that  the 
number  of  UCAVs  that  a  single  operator  can  control  is  not  just  a  function  of  the  level  of  decision  support 
automation,  but  is  inextricably  tied  to  both  mission  complexity  and  overall  system  performance  [24], 
Using  human  experimentation  in  a  multiple  UCAV  simulation  test  bed  and  a  simulated  annealing  (SA) 
technique  for  heuristic -based  optimization,  operator  performance  was  predicted  to  be  significantly 
degraded  beyond  approximately  five  UCAVs,  with  an  optimal  bound  between  2-4  vehicles  (Figure  5).  The 
KPP  in  this  model  was  cost,  which  took  into  account  not  just  operational  costs  such  as  fuel,  but  also  the 
cost  of  missed  targets  and  cost  in  terms  of  mission  delays  introduced  by  inefficient  human  interactions. 
The  solid  curve  in  Figure  5  represents  a  theoretically  perfect  human  operator,  and  the  dotted  line 
represents  more  realistic  human  performance  that  accounts  for  delays  due  to  inefficient  decision  making, 
communication  problems,  cognitive  load,  etc.  Thus,  the  performance  of  the  system  (the  automation  and 
the  operator)  can  vary  both  as  a  function  of  the  operator,  but  also  can  vary  due  to  the  operational 
constraints  such  as  number  of  targets,  operational  costs,  etc.  This  variation  is  why  it  is  important  to 
explicitly  link  system  performance  to  operator  capacity. 
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4.4  Reliability  and  Trust 

Human  operator  interaction  and  performance  using  automated  decision  support  agents  have  been  shown  to 
be  significantly  influenced  by  system  reliability  and  resultant  (dis)trust,  i.e.,  the  more  reliable  automation 
is,  the  more  trust  users  have  and  the  more  likely  they  are  to  use  the  automation  [35-37].  Conversely, 
systems  with  poor  or  inconsistent  reliability  will  often  be  disregarded  [38,  39].  This  is  especially  true  in 
C2  supervisory  control  systems  where  operators  are  multitasking  while  monitoring  various  mission- 
critical  automated  systems.  High  reliability  may  lead  to  overtrust,  automation  bias,  complacency,  and  loss 
of  situation  awareness.  Degraded  reliability  can  lead  to  undertrust,  misuse,  and  disuse  of  automation  [12, 
38,40,41], 

Of  the  studies  reviewed  here,  only  a  few  examined  the  impact  of  trust  and  reliability.  Addressing 
reliability,  Ruff  et  al.  [10,  20]  concluded  that  performance  decreases  observed  in  low  reliability  conditions 
was,  in  part,  due  to  lower  trust  in  the  automated  decision-aid  provided  to  the  human  operator.  This  was 
especially  true  as  the  number  of  vehicles  per  operator  increased.  Examining  trust,  Dixon  and  Wickens  et 
al.  [16,  19]  investigated  the  effect  of  perfectly  reliable  versus  degraded  automation  on  performance  in 
multi-UAV  monitoring  tasks.  They  concluded  that  reliable  automation  could  alleviate  task  interference 
and  reduce  workload,  and  allow  a  single -operator  to  control  several  UAVs.  However,  even  perfectly 
reliable  automation  could  not  prevent  a  decrease  in  performance  when  workload  increased. 

4.5  Broader  Command  and  Control  Implications 

While  the  focus  of  this  paper  is  primarily  on  human  supervisory  control  of  multiple  UASs,  there  are  many 
parallels  across  more  general  command  and  control  settings,  particularly  for  network  centric  operations. 
At  a  more  fundamental  level,  the  multiple  vehicle  supervisory  control  paradigm  represents  an  attention 
allocation  problem  which  requires  an  operator  to  determine  how  and  when  to  allocate  limited  cognitive 
resources  to  multiple,  often  competing  tasks.  Thus  a  single  operator  controlling  multiple  unmanned 
vehicles  is  analogous  to  command  and  control  tasks  in  time -pressured,  uncertain,  and  dynamic  settings 
that  concurrently  compete  for  operator  attention.  For  example,  in  many  cases  there  is  a  negative  impact  on 
primary  task  performance  when  military  personnel  attempt  to  attend  to  multiple  chat  streams,  particularly 
in  operations  centers  [42-44],  indicating  that  operators  are  struggling  with  attention  allocation,  especially 
in  task  prioritization. 
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For  the  multiple  unmanned  vehicle  problem,  as  well  as  more  general  problems  like  chat  management, 
operator  capacity  may  ultimately  be  limited  by  the  heterogeneity,  as  opposed  to  being  strictly  limited  by 
the  number  of  vehicles/tasks  under  control.  Research  is  currently  underway  to  determine  the  impact  of 
task  heterogeneity  as  opposed  to  vehicle  and  payload  heterogeneity  [45],  as  well  as  the  development  of  a 
class  of  metrics  to  measure  attention  allocation  efficiency  [46]. 

The  move  towards  network  centric  operations  will  mean  that  operators  must  deal  with  many  competing 
sources  of  incoming  information,  which  could  easily  overload  their  cognitive  bandwidth.  Thus,  research 
investigating  operator  capacity  limits  both  from  a  vehicle  and  task  heterogeneity  perspective,  is  critical  not 
just  for  multiple  unmanned  vehicle  control,  but  also  for  those  command  and  control  settings  which  require 
operators  to  make  critical  decisions  with  multiple  sources  of  time-sensitive  information. 

5.0  CONCLUSION 

In  response  to  the  OSD  UAS  Roadmap  and  the  future  vision  of  NCW,  a  number  of  defense  industries  are 
now  investigating  and  developing  possible  platforms  and  workstations  to  enable  single  operator  control  of 
multiple  UAVs.  For  example,  QinetiQ  recently  demonstrated  that  multiple  self-organizing  UAVs  can  be 
effectively  controlled  by  an  operator  flying  of  a  fast  jet.  Lockheed  Martin  has  developed  a  prototype 
control  station  that  not  only  allows  for  single  operator  control  of  multiple  UAVs,  but  unmanned  ground 
vehicles  as  well.  Raytheon’s  Universal  Control  System  is  advertised  as  providing  single  operator  control 
of  multiple  UASs,  simultaneously. 

Despite  the  aggressive  development  of  these  multiple  UAV/UAS  control  technologies,  as  we  have 
demonstrated  here,  the  research  community  is  only  beginning  to  grasp  the  nuances  of  human  interaction  in 
the  cognitively  demanding  environment  of  multiple  vehicle  control.  While  this  meta-analysis  suggests 
promising  trends  in  that  with  increasing  automation  across  all  three  control  loops,  it  is  likely  that  operators 
can  control  an  increasing  number  of  UAVs,  the  actual  number  will  depend  on  a  variety  of  other  factors 
(system  reliability,  communication  bandwidth,  mission  context,  complexity  and  operational  tempo, 
operator  training,  etc.). 

In  addition,  as  previously  discussed,  one  serious  drawback  to  the  insertion  of  high  levels  of  autonomy  in 
the  command  and  control  of  UAVs  is  the  strong  possibility  that  automation  bias  could  lead  to 
complacency  and  erroneous,  if  not  catastrophic  outcomes.  This  decision  bias  has  not  only  been  seen  in 
many  experimental  command  and  control  settings  [15,  21],  but  also  in  reality  in  the  recent  Gulf  War 
conflict  [14].  Thus  this  possible  negative  outcome  of  increasing  autonomy  presents  designers  of  both  the 
technology  as  well  as  the  encompassing  organizational  structures  with  a  challenge:  How  to  design  reliable 
and  trustworthy  automated  systems  that  reduce  operator  workload,  but  also  keep  their  situation  awareness 
sufficiently  high  so  that  they  can  recognize  and  intervene  when  automation  fails?  It  remains  to  be  seen 
whether  the  systems  in  development  today  will  actually  account  for  all  of  these  socio-technical  variables 
instead  of  just  focusing  on  the  technical  ones. 

The  critical  lesson  to  be  learned  from  this  meta-analysis  is  that  the  success  of  any  UAS,  and  more 
generally  multiple  command  and  control  task  management,  is  not  just  contingent  on  high  levels  of 
autonomy,  but  more  linked  to  robust  system  automation  strategies  that  account  for  human  operators’ 
cognitive  abilities,  both  positive  and  negative.  Thus  it  is  critical  that  developers  of  multiple  UAS  control 
stations  today  recognize  and  design  for  the  complex  interactions  between  highly  automated  systems  and 
the  need  to  support  human  knowledge -based  reasoning. 

6.0  ACKNOWLEDGMENTS 

This  research  is  partially  funded  by  the  Office  of  Naval  Research.  The  views  and  conclusions  presented  in 
this  paper  are  those  of  the  authors  and  do  not  represent  an  official  opinion,  expressed  or  implied,  of  ONR. 
Special  thanks  to  Sally  Chapman  for  her  support  of  this  research. 

This  work  initially  appeared  in  the  International  Command  and  Control  Journal ,  (2007)  1(2),  p.  1-24. 


1  -10 


RTO-EN-SCI-208 


UNCLASSIFIED/UNLIMITED 


UNCLASSIFIED/UNLIMITED 

Automation  Architecture  for  Single  Operator-Multiple  UAV 

Command  and  Control 


7.0  REFERENCES 

[1]  Office  of  the  Secretary  of  Defense,  "Unmanned  Aircraft  Systems  (UAS)  Roadmap,  2005-2030," 
DoD,  Washington  DC  2005. 

[2]  Naval  Studies  Board,  "Autonomous  Vehicles  in  Support  of  Naval  Operations,"  National  Research 
Council,  Washington  DC  2005. 

[3]  D.  S.  Alberts,  J.  J.  Garstka,  and  F.  P.  Stein,  Network  Centric  Warfare.  Washington  DC:  DoD 
Command  and  Control  Research  Program,  1999. 

[4]  D.  S.  Alberts  and  R.  E.  Hayes,  Understanding  Command  and  Control.  Washington  DC:  DoD 
Command  and  Control  Research  Program,  2006. 

[5]  R.  J.  Curts  and  J.  P.  Frizzell,  "Implementing  Network-Centric  Command  and  Control,"  presented 
at  10th  International  Command  and  Control  Research  and  Technology  Symposium,  Washington 
DC,  2005. 

[6]  R.  S.  Bolia,  M.  A.  Vidulich,  and  W.  T.  Nelson,  "Unintended  Consequences  of  the  Network- 
Centric  Decision  Making  Model:  Considering  the  Human  Operator,"  presented  at  Command  and 
Control  Research  and  Technology  Symposium,  San  Diego,  CA,  2006. 

[7]  T.  B.  Sheridan,  Telerobotics,  Automation  and  Human  Supervisory  Control.  Cambridge,  MA:  The 
MIT  Press,  1992. 

[8]  J.  Rasmussen,  "Skills,  Rules,  and  Knowledge;  Signals,  Signs,  and  Symbols,  and  Other 
Distractions  in  Human  Performance  Models,"  IEEE  Transactions  on  Systems,  Man,  and 
Cybernetics,  vol.  SMC- 13,  pp.  257-266,  1983. 

[9]  R.  Parasuraman,  T.  B.  Sheridan,  and  C.  D.  Wickens,  "A  Model  for  Types  and  Uevels  of  Human 
Interaction  with  Automation,"  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics  -  Part  A: 
Systems  and  Humans,  vol.  30,  pp.  286-297,  2000. 

[10]  H.  A.  Ruff,  S.  Narayanan,  and  M.  H.  Draper,  "Human  Interaction  with  Levels  of  Automation  and 
Decision-Aid  Fidelity  in  the  Supervisory  Control  of  Multiple  Simulated  Unmanned  Air  Vehicles," 
Presence,  vol.  11,  pp.  335-351,  2002. 

[1 1]  M.  L.  Cummings,  A.  S.  Brzezinski,  and  J.  D.  Lee,  "The  Impact  of  Intelligent  Aiding  for  Multiple 
Unmanned  Aerial  Vehicle  Schedule  Management,"  IEEE  Intelligent  Systems,  in  press. 

[12]  M.  L.  Cummings  and  P.  J.  Mitchell,  "Automated  Scheduling  Decision  Support  for  Supervisory 
Control  of  Multiple  UAVs,"  Journal  of  Aerospace  Computing,  Information,  and  Communication, 
in  press. 

[13]  K.  L.  Mosier  and  L.  J.  Skitka,  "Human  Decision  Makers  and  Automated  Decision  Aids:  Made  for 
Each  Other?,"  in  Automation  and  Human  Performance:  Theory  and  Applications,  Human  Factors 
in  Transportation,  R.  Parasuraman  and  M.  Mouloua,  Eds.  Mahwah,  New  Jersey:  Lawrence 
Erlbaum  Associates,  Inc.,  1996,  pp.  201-220. 

[14]  32nd  Army  Air  and  Missile  Defense  Command,  "Patriot  Missile  Defense  Operations  during 
Operation  Iraqi  Freedom,"  U.  S.  Army,  Ed.  Washington  DC,  2003. 

[15]  M.  L.  Cummings,  "Automation  Bias  in  Intelligent  Time  Critical  Decision  Support  Systems," 
presented  at  AIAA  Intelligent  Systems,  Chicago,  IL,  2004. 

[16]  S.  Dixon,  C.  Wickens,  and  D.  Chang,  "Mission  Control  of  Multiple  Unmanned  Aerial  Vehicles:  A 
Workload  Analysis,"  Human  Factors,  in  press. 

[17]  A.  T.  Welford,  "The  psychological  refractory  period  and  the  timing  of  high-speed  performance  -  a 
review  and  a  theory,"  British  Journal  of  Psychology,  vol.  43,  pp.  2-19,  1952. 

[18]  D.  E.  Broadbent,  Perception  and  Communication.  Oxford:  Pergamon,  1958. 

[19]  S.  Dixon,  C.  D.  Wickens,  and  D.  Chang,  "Unmanned  Aerial  Vehicle  Flight  Control:  False  Alarms 
Versus  Misses,"  presented  at  Humans  Factors  and  Ergonomics  Society  48th  Annual  Meeting,  New 
Orleans,  2004. 


RTO-EN-SCI-208 


1  - 11 


UNCLASSIFIED/UNLIMITED 


UNCLASSIFIED/UNLIMITED 


Automation  Architecture  for  Single  Operator-Multiple  UAV 
Command  and  Control 


mrt  sgAtftmf 


[20]  H.  A.  Ruff,  G.  L.  Calhoun,  M.  H.  Draper,  J.  V.  Fontejon,  and  B.  J.  Guilfoos,  "Exploring 
Automation  Issues  in  Supervisory  Control  of  Multiple  UAVs,"  presented  at  2nd  Human 
Performance,  Situation  Awareness,  and  Automation  Conference  (HPSAA  II),  Daytona  Beach,  FL, 
2004. 

[21]  R.  D.  Dunlap,  "The  evolution  of  a  distributed  command  and  control  architecture  for  semi- 
autonomous  air  vehicle  operations,"  presented  at  Moving  Autonomy  Forward  Conference, 
Grantham,  UK,  2006. 

[22]  M.  L.  Cummings  and  S.  Guerlain,  "Developing  Operator  Capacity  Estimates  for  Supervisory 
Control  of  Autonomous  Vehicles,"  Human  Factors,  vol.  49,  pp.  1-15,  2007. 

[23]  M.  L.  Cummings  and  Mitchell  P.J.,  "Predicting  Controller  Capacity  in  Remote  Supervision  of 
Multiple  Unmanned  Vehicles,"  IEEE  Systems,  Man,  and  Cybernetics,  Part  A  Systems  and 
Humans,  in  press. 

[24]  M.  L.  Cummings,  C.  E.  Nehme,  .,  and  J.  Crandall,  "Predicting  Operator  Capacity  for  Supervisory 
Control  of  Multiple  UAVs,"  in  Innovations  in  Intelligent  Machines  vol.  70,  Studies  in 
Computational  Intelligence  J.  S.  Chahl,  L.  C.  Jain,  A.  Mizutani,  and  M.  Sato-Ilic,  Eds.,  2007. 

[25]  M.  Lewis,  J.  Polvichai,  K.  Sycara,  and  P.  Scerri,  "Scaling-up  Human  Control  for  Large  UAV 
Teams,"  in  Human  Factors  of  Remotely  Operated  Vehicles,  N.  Cooke,  H.  Pringle,  H.  Pedersen, 
and  O.  Connor,  Eds.  New  York:  Elsevier,  2006,  pp.  237-250. 

[26]  M.  R.  Endsley  and  D.  B.  Kaber,  "Level  of  automation  effects  on  performance,  situation  awareness 
and  workload  in  a  dynamic  control  task,"  Ergonomics,  vol.  42,  pp.  462-492,  1999. 

[27]  C.  D.  Wickens,  S.  E.  Gordon,  and  Y.  Liu,  An  Introduction  to  Human  Factors  Engineering.  New 
York:  Longman,  1998. 

[28]  M.  R.  Endsley,  "Toward  a  Theory  of  Situation  Awareness  in  Dynamic  Systems,"  Human  Factors, 
vol.  37,  pp.  32-64,  1995. 

[29]  T.  B.  Sheridan  and  W.  L.  Verplank,  "Human  and  Computer  Control  of  Undersea  Teleoperators," 
MIT,  Cambridge,  Man-Machine  Systems  Laboratory  Report  1978. 

[30]  B.  Hilburn,  P.  G.  Jorna,  E.  A.  Byrne,  and  R.  Parasuraman,  "The  Effect  of  Adaptive  Air  Traffic 
Control  (ATC)  Decision  Aiding  on  Controller  Mental  Workload,"  in  Human-automation 
Interaction:  Research  and  Practice.  Mahwah,  NJ:  Lawrence  Erlbaum,  1997,  pp.  84-91. 

[31]  P.  J.  Smith,  E.  McCoy,  and  C.  Layton,  "Brittleness  in  the  design  of  cooperative  problem-solving 
systems:  the  effects  on  user  performance,"  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics, 
vol.  27,  pp.  360-370,  1997. 

[32]  M.  R.  Endsley  and  D.  J.  Garland,  Situation  Awareness  Analysis  and  Measurement.  Mahwah,  NJ: 
Lawrence  Erlbaum  Associates,  Inc.,  2000. 

[33]  H.  A.  Simon,  R.  Hogarth,  C.  R.  Piott,  H.  Raiffa,  K.  A.  Schelling,  R.  Thaier,  A.  Tversky,  and  S. 
Winter,  "Decision  Making  and  Problem  Solving,"  presented  at  Research  Briefings  1986:  Report  of 
the  Research  Briefing  Panel  on  Decision  Making  and  Problem  Solving,  Washington  D.C.,  1986. 

[34]  Joint  Chiefs  of  Staff,  "Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  6212.01D,"  DoD,  2007. 

[35]  J.  D.  Lee  and  N.  Moray,  "Trust,  Control  Strategies  and  Allocation  of  Functions  in  Human- 
Machine  Systems,"  Ergonomics,  vol.  35,  pp.  1234-1270,  1992. 

[36]  J.  D.  Lee  and  N.  Moray,  "Trust,  Self-Confidence,  and  Operators'  Adaptation  to  Automation," 
International  Journal  of  Human-Computer  Studies,  vol.  40,  pp.  153-184,  1994. 

[37]  R.  Parasuraman,  "Effects  of  adaptive  function  allocation  on  human  performance,"  in  Human 
factors  and  advanced  aviation  technologies,  D.  J.  Garland  and  J.  A.  Wise,  Eds.  Daytona  Beach, 
FL:  Embry-Riddle  Aeronautical  University  Press,  1993,  pp.  147-157. 

[38]  R.  Parasuraman  and  V.  A.  Riley,  "Humans  and  automation:  Use,  misuse,  disuse,  and  abuse.," 
Human  Factors,  vol.  39,  pp.  230  -  253,  1997. 

[39]  V.  Riley,  "A  theory  of  operator  reliance  of  automation,"  in  Human  performance  in  automated 
systems:  Current  research  and  trends,  M.  Mouloua  and  R.  Parasuraman,  Eds.  Hillsdale,  NJ: 
Lawrence  Erlbaum  Associates,  1994,  pp.  8-14. 

[40] 


1  - 12 


RTO-EN-SCI-208 


UNCLASSIFIED/UNLIMITED 


UNCLASSIFIED/UNLIMITED 

Automation  Architecture  for  Single  Operator-Multiple  UAV 

Command  and  Control 


M.  L.  Cummings,  "Automation  Bias  in  Intelligent  Time  Critical  Decision  Support  Systems,"  presented  at 
AIAA  3rd  Intelligent  Systems  Conference,  Chicago,  2004. 

[41]  R.  Parasuraman,  R.  T.  Molloy,  and  I.  L.  Singh,  "Performance  consequences  of  automation- 
induced  "complacency","  International  Journal  of  Aviation  Psychology,  vol.  3,  1993. 

[42]  D.  Caterinicchia,  "DoD  Chat  Use  Exploded  in  Iraq,"  in  Federal  Computer  Week,  2003. 

[43]  L.  Boiney,  "Team  Decision  Making  in  Time-Sensitive  Environments,"  presented  at  10th 
International  Command  and  Control  Research  and  Technology  Symposium,  McLean,  Virginia, 
2005. 

[44]  N.  J.  Heacox,  "Survey  of  Chat  Usage  in  the  Fleet  Results,"  Pacific  Science  &  Engineering  Group, 
San  Diego,  CA  2003. 

[45]  R.  M.  Kilgore,  K.  A.  Harper,  C.  Nehme,  and  M.  L.  Cummings,  "Mission  Planning  and  Monitoring 
for  Heterogeneous  Unmanned  Vehicle  Teams:  A  Human-Centered  Perspective,"  presented  at 
AIAA  Infotech@  Aerospace  Conference,  Sonoma,  CA,  2007. 

[46]  J.  W.  Crandall  and  M.  L.  Cummings,  "Identifying  Predictive  Metrics  for  Supervisory  Control  of 
Multiple  Robots,"  IEEE  Transactions  on  Robotics  -  Special  Issue  on  Human-Robot  Interaction, 
accepted. 


RTO-EN-SCI-208 


1  -13 


UNCLASSIFIED/UNLIMITED 


UNCLASSIFIED/UNLIMITED 


Automation  Architecture  for  Single  Operator-Multiple  UAV 
Command  and  Control 


1  - 14 


RTO-EN-SCI-208 


UNCLASSIFIED/UNLIMITED 


